Evaluation Framework for Fuzzy Theoretic-Based Recommender System

Empirical evaluation of user model based adaptive systems is challenging and intriguing due to the nature of user model and adaptive decision processes. Their success also depends on the subjective judgment of the user. It can be done by evaluating the constructed user model and the resulting adaptive decision made based upon the user model. Until recently, literature on comprehensive empirical evaluations of user modeling systems and adaptive systems are rare. The main reason is the lack of an evaluation methodology. Emerging literature attempts to address this gap in two fronts: by developing an evaluation methodology and by evaluating the user adaptive system. In this paper various evaluation methodologies are reviewed and adapted to fit the evaluation requirements of the fuzzy set theoretic user model driven recommender systems. Another related problem for evaluating the effectiveness of user models and adaptive recommender systems with regards to performance and usability is the absence of clear and effective evaluation metrics. In this paper evaluation metrics from the information retrieval community are adapted and discussed to evaluate effectiveness of the fuzzy theoretic-based adaptive recommender systems. Finally, in order to illustrate the application of the proposed evaluation framework, a simulation system for Fuzzy Theoretic-based Movie Recommender System is designed, developed and evaluated using statistical methods.